AD-A193  102 


AFHRL-TP-87-62  OTIC  FILE 


AIR  FORCE 


PERSONALITY,  ATTITUDES,  AND  PILOT  TRAINING 
PERFORMANCE:  PRELIMINARY  ANALYSIS 


Frederick  M.  Slem 
Thomas  R.  Carretta 
Theresa  A.  Mercatante 


AIR  FORCE  SYSTEMS  COMMAND 
BROOKS  AIR  FORCE  BASE,  TEXAS  78235-5801 


NOTICE 


When  Government  drawings,  specifications,  or  other  data  are  used  for  any 
purpose  other  than  In  connection  with  a  definitely  Government-related 
procurement,  the  United  States  Government  Incurs  no  responsibility  or  any 
obligation  whatsoever.  The  fact  that  the  Government  may  have  formulated  or 
In  any  way  supplied  the  said  drawings,  specifications,  or  other  data.  Is 
not  to  be  regarded  by  implication,  or  otherwise  In  any  manner  construed,  as 
licensing  the  holder,  or  any  other  person  or  corporation;  or  as  conveying 
any  rights  or  permission  to  manufacture,  use,  or  sell  any  patented 
invention  that  may  in  any  way  be  related  thereto. 


The  Public  Affairs  Office  has  reviewed  this  paper,  and  It  is  releasable  to 
the  National  Technical  Information  Service,  where  It  will  be  available  to 
the  general  public.  Including  foreign  nationals. 

This  paper  has  been  reviewed  and  1$  approved  for  publication. 


WILLIAM  E.  ALLEY,  Technical  Director 
Manpower  and  Personnel  Division 


DANIEL  L.  LEIGHTON,  Lt  Col,  USAF 
Chief,  Manpower  and  Personnel  Division 


Unclassified 


SECURITY  CLASSIFICATION  OFTHlSJAGE _ 

REPORT  DOCUMENTATION  PAGE 


lb.  RESTRICTIVE  MARKINGS 


Form  Approved 
OMB  No.  07044188 


la.  REPORT  SECURITY  CLASSIFICATION 
Unclassified 


2a.  SECURITY  CLASSIFICATION  AUTHORITY 


2b.  DECLASSIFICATION /DOWNGRADING  SCHEDULE 


4.  PERFORMING  ORGANIZATION  REPORT  NUMBER(S) 
AFHRL-TP-87-62 


3  .  DISTRIBUTION  /AVAILABILITY  OF  REPORT 

Approved  for  public  release;  distribution  Is  unlimited. 


5.  MONITORING  ORGANIZATION  REPORT  NUMBER(S) 


6a.  NAME  OF  PERFORMING  ORGANIZATION 


6b.  OFFICE  SYMBOL  7a.  NAME  OF  MONITORING  ORGANIZATION 
(If  applicable) 


Manpower  and  Personnel  Division  j  AFHRL/MOEA 
6c  ADDRESS  ( Gty ,  State,  and  ZIP  Code) 

Air  Fore  •  Human  Resources  laboratoiy 
Brooks  Air  Force  Base,  Texas  78235-5601 


7b.  ADDRESS  (O'ty,  State,  and  ZIP  Code) 


8a.  NAfE  OF  FUNOING/  SPONSORING 
ORGANIZATION 


8b!  OFFICTTsYMBOl"  '  J  9"  PROCUREMENT  INSTRUMENT  IDENTIFICATION  NUMBER 
(If  applicable)  | 


Air  Force  Human  Resources  Laboratory  | 

8c.  ADDRESS  (City,  State,  and  ZIP  Code) 

Brooks  Air  Force  Base,  Texas  78235-*5601 


HQ  AFHRL 


10.  SOURCE  OF  FUNDING  NUMBERS 

PROGRAM  I  PROJECT  fl 

ELEMENT  NO.  I  NO.  I 


PROJECT 

TASK 

NO. 

NO 

7719 

18 

WORK  UNIT 
ACCESSION  N 

45 


11.  TITLE  (Include  Security  Classification) 

!  Personality,  Attitudes,  and  Pilot  Training  Performance:  Preliminary  Analysis 


12.  PERSONAL  AUTHOR(S) 

Slem,  F.M.;  Carretta,  T.R.;  Mercatante,  T.A. 


13a.  TYPE  OF  REPORT 
Interim 


13b.  TIME  COVERED 
FROM  Jan  83  TO  Jan  87 


14.  DATE  OF  REPORT  (Year,  Month,  Day)  15.  PAGE  COUNT 

_ May  1988 _  20 


16.  SUPPLEMENTARY  NOTATION 

Paper  presented  at  the  Third  Annual  South  Texas  Symposium  on  Human  Factors  and  Ergonomics  sponsored  by  the  Alamo 
Chapter  of  the  Human  Factors  Society,  University  of  Texas  at  San  Antonio,  San  Antonio.  Texas.  8  Mav  1987. 

17.  COSATI  CODE)  18.  SUBJECT  TERMS  (Continue  on  reverse  if  necessary  and  identify  by  block  number) 

field  group  SUB-GROUP  attitude  assessment  Individual  differences 

•  /  nc  ~  Basic  Attributes  Tests  personality  assessment 

ns  no  _c1ass1f1cat1on  pilot  selection 

j  19  ABSTRACT  (Continue  on  reverse  if  necessary  and  identify  by  block  number) 

|  '""Developments  In  research  concerning  personality  characteristics  have  led  to  a  renewed  Interest  In 
j  applications  of  Individual  differences  measures  for  selection  of  pilot  candidates.  Recent  research  efforts  have 

1  focused  on  selecting  for  positive  characteristics,  rather  than  screening  out  pathological  traits.  Another 

j  development  is  the  use  of  tests  in  which  the  dimension  of  Interest  is  not  readily  apparent  to  the  examinee. 

I  In  the  present  study,  five  personality  and  attitude  measures  were  adalnlstered  to  883  USAF  pilot  candidates  as 
|  part  of  an  experimental  test  battery  under  consideration  for  operational  use  In  pilot  selection  and 

j  classification.  These  tests  were  designed  to  assess  decisiveness,  risk-taking,  self-confidence,  survival 

'  attitudes,  and  field  dependence/Independence.  Scores  from  these  tests  wer*  examined  for  their  utility  In 

predicting  training  outcome  (graduation  or  elimination)  and  advanced  training  recommendation  (fighter  or 
non-fighter  aircraft). 

j  (Continued) 7 


i  20  DISTRIBUTION  /AVAILABILITY  OF  ABSTRACT  <i.-» 

j  UDUNCLASSIFIEO/UNLIMITEO  □  SAME  AS  RPT.  □  OTIC  USERS 
j  22a  NAME  OF' RESPONSIBLE  INDIVIDUAL  22b.  TEL 

1  Haney  J.  All  In.  Chief.  STINFO  Office  ( 

DO  Form  1473,  JUN  36  Previous  editions  are  obsolete. 


21.  ABSTRACT  SECURITY  CLASSIFICATION 

22b.  TELEPHONE  (Include  A>  “  Code)  I  22c.  OFFICE  SYMBOL 


SECURITY  CLASSIFICATION  OF  THIS  PAGE 
Unclassified 


(Concluded) 


Results  Indicated  that  as  a  group,  the  tests  demonstrated  weak  relationships  with  the 
performance  criteria.  No  test  manifested  a  consistent  pattern  of  validity  for  both 
pe-formance  measures.  Only  the  test  of  self-confidence  appeared  to  contribute  to  predicting 
completion  of  training.  Future  research  efforts  are  discussed  with  regard  to  refining  the 
current  test  of  self-confidence  and  establishing  Its  construct  validity. 


1 


AFHRL  Technical  Paper  87-62 


May  1988 


PERSONALITY,  ATTITUDES,  AND  PILOT  TRAINING 
PERFORMANCE:  PRELIMINARY  ANALYSIS 


Frederick  M.  Sien 
Thomas  R.  Carretta 
Theresa  A.  Mercatante 


NANPOVER  AND  PERSONNEL  DIVISION  • 
8 rooks  Air  Force  Base,  Texas  78235-5601 


j  Aooeaslon  For 
NTIS  GRA&I 
MIC  TAB 
Unannounced 
Justification 


□ 


By, 


Distribution/ 


j  ^Availability  Codes 
Avei’L  and/or 
Dist  I  Special 


M 


Reviewed  and  submitted  for  publication  by 
Lloyd  D.  Burtch 

Chief,  Cognitive  Skills  Assessment  Branch 


Paper  presented  at  the  Third  Annual  South  Texas  Symposlw  on  Human  Factors  and  Ergonomics 
sponsored  by  the  Alamo  Chapter  of  the  Human  Factors  Society,  University  of  Texas  at  San  Antonio, 
San  Antonio,  Texas,  8  May  1987. 


SIMMRY 


A  computer-administered  test  battery,  the  Basic  Attributes  Tests  (BAT),  was  administered  to 
883  USAF  pilot  candidates.  Five  tests  designed  to  assess  personality  and  attltudlnal 
characteristics  (decisiveness,  risk-taking,  self-confidence,  survival  attitudes,  and  field 
dependence/Independence)  were  examined  for  their  utility  In  predicting  training  outcome 
(graduation  or  elimination)  and  recowendatlon  for  advanced  training  assignment  (fighter  or 
non-fighter  aircraft). 

Results  Indicated  that  although  the  tests  demonstrated  acceptable  reliability,  as  a  group 
they  were  not  strongly  related  with  the  performance  criteria.  None  of  the  tests  was  valid 
against  both  performance  measures.  Only  the  test  of  self-confidence  appeared  to  contribute  to 
predicting  completion  of  training.  Future  research  efforts  are  discussed  with  regard  to  refining 
the  current  test  of  self-confidence  and  establishing  Its  construct  validity. 
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PREFACE 


This  project  was  conducted  under  Work  Unit  77191845.,  Selection  for  Undergraduate 
Pilot  Training,  Issued  by  the  USAF  Air  Training  Conwand.  This  paper  was  presented  at 
the  Third  Annual  South  Texas  Symposium  on  Hunan  Factors  and  Ergonomics  sponsored  by  the 
Alamo  Chapter  of  the  Hunan  Factors  Society.  The  symposium  took  place  at  the  University 
of  Texas  at  San  Antonio  on  8  May  1987. 

This  paper  Is  Intended  to  serve 'as  an  Interim  report  regarding  five  tests  of 
personality  and  attltudlnal  characteristics  from  the  Basic  Attributes  Tests  (BAT) 
battery,  being  considered  for  use  as  selection  and  classification  Instruments  for 
aircrew  candidates. 
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PERSONALITY,  ATTITUDES,  AND  PILOT  TRAINING 
PERFORMANCE:  PRELIMINARY  ANALYSIS 


I.  INTRODUCTION 

Individual  differences  in  student  motivation,  personality,  and  attitudes  have  been 
acknowledged  as  Important  determinants  of  successful  performance  In  the  training  environment. 
The  Air  Force  has  a  long  history  of  Interest  In  this  field  of  research,  particularly  with  regard 
to  the  Issue  of  officer  candidate  selection  (e.g..  Flyer  &  Blgbee,  1955;  Guinn,  Vltola,  4  Lelsey, 
1976;  Mullins,  1962;  Taylor,  Murray,  Ellison,  4  Majesty,  1.971;  Tupes  4  Chrlstal,  1957).  A 
major  factor  contributing  to  this  concern  Is  the  high  costs  associated  with  candidate  attrition, 
particularly  In  Undergraduate  Pilot  Training  (UPT).  The  cost  for  each  UPT  ellmlnee  currently  Is 
estimated  to  be  about  $65,000  to  $80,000. 

To  put  curre- 1  pilot  selection  research  efforts  Into  context,  a  brief  background  history 
follows.  Research  concerning  the  selection  of  military  pilot  candidates  dates  back  to  World  War 

I.  Efforts  at  Kelly  Field  In  1919  found  the  highest  predictive  validity  for  pilot  selection  with 
measures  of  mental  alertness  and  emotional  stability  (North- 4  Griffin,  1977).  During  World  War 

II,  the  emphasis  on  pilot  selection  focused  on  apparatus  tests  that  were  used  to  measure 
performance  In  terms  of  two-hand  coordination,  stlck-and-rudder  skills,  rotary  pursuit  with 
divided  attention,  and  discrimination  reaction  time.  However,  problems  with  mechanical 
reliability  of  the  apparatus  devices,  and  a  policy  shift  toward  decentralized  testing  In  1955, 
led  to  the  discontinuation  of  apparatus  testing  in  favor  of  paper-and-pencll  tests  (Bordelon  4 
Kantor,  1986). 

Several  research  efforts  over  the  next  two  decades  were  designed  to  Increase  the  validity  of 
paper-and-pencll  tests  for  pilot  selection  through  the  Inclusion  of  measures  of  motivation, 
occupational  Interests,  and  personality  characteristics  (e.g.,  Guinn  et  al.,  1976;  Mullins, 
1962;  Taylor  et  al.,  1971).  Although  these  studies  provided  modest  support  for  the  use  of 
Interest  and  motivation  measures  for  pilot  selection,  such  tests  never  were  put  into  operational 
use. 


Technological  developments  In  the  1970s  led  to  a  renewed  interest  In  apparatus  testing  for 
pilot  selection  (Hunter  &  Thompson,  1978;  Long  4  Varney,  1975;  McGrevy  4  Valentine,  1974).  The 
result  was  development  of  a  computer-administered  test  battery,  the  Basic  Attributes  Tests 
System,  or  BAT  (Bordelon  4  Kantor,  1986).  The  battery  Includes  a  nianber  of  tests  designed  to 
measure  characteristics  and  skills  Identified  as  having  potential  for  predicting  pilot 
performance  (Imhoff  4  Levine,  1981).  Included  among  the  tests  are  several  tests  validated  In 
World  War  II  at  the  Army  Air  Forces  School  of  Aviation  Medicine,  such  as  measures  of  psychomotor 
performance  and  reaction  time  (Melton,  1947).  Other  tests  Included  In  the  battery  are  based  on 
more  recent  advances  In  the  disciplines  of  cognitive  psychology  and  personality  assessment 
(Imhoff  4  Levine,  1981).  The  present  focus  Is  on  the  predictive  utility  of  personality  and 
attitude  measures  Included  in  the  battery. 

Five  tests  from  the  BAT  will  be  discussed.  Two  of  these  tests  represent  computerized 
adaptations  of  paper-and-pencll  tests  used  previously  In  Air  Force  research.  Dot  Estimation,  for 
example,  was  developed  to  measure  Impulsiveness;  and  Self-Crediting  Word  Knowledge  had  been  used 
as  a  test  of  self-confidence  (Mullins,  1962).  Other  tests  were  selected  to  measure 
characteristics  such  as  field  dependence/Independence,  risk-taking  behavior,  and  attitudes  about 
risk-taking.  These  tests  were  either  designed  or  selected  on  the  basis  of  their  content  validity 
for  pilot  research.  However,  It  should  be  noted  that  the  tests  selected  from  the  literature  had 
previously  been  used  mainly  In  academic  settings,  for  purposes  other  than  aircrew  selection. 
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Based  on  previous  research  and  anecdotal  data.  It  was  expected  that  pilot  candidates  who 
successfully  completed  training  would  be  less  Impulsive,  more  self-confident,  and  more  field 
Independent;  It  also  was  expected  that  successful  candidates  would  manifest  differences  In 
decision-making  style  and  In  attitudes  about  risk-taking  behavior,  relative  to  candidates  who 
failed  to  graduate  from  pilot  training.  Finally,  It  was  expected  that  within  the  group  of  pilots 
who  completed  training  successfully,  there  would  be  differences  In  characteristics  and  attitudes 
between  two  groups  of  students:  those  recommended  for  advanced  training  In  a 
fighter-attack-reconnaissance  (FAR)  track,  and  those  recooaended  for  advanced  training  In  a 
tanker-transport-bomber  track  (TTB). 


II.  METHOD 


Subjects 


The  sample  consisted  of  883  trainees  In  the  USAF  Undergraduate  Pilot  Training  (UPT)  program, 
80-  males  and  16  females.  In  terms  of  race,  the  sample  consisted  of  811  Caucasians,  13  Blacks, 
and  two  'Other*  (with  no  data  available  on  57  subjects).  The  mean  age  for  the  sample  was  23.8 
years.  The  composition  of  the  sample  In  terms  of  coonisslonlng  source  Is  shown  In  Table  1. 


Table  1.  Distribution  of  Subjects  by  Source  (H  »  883) 


Source 

FSP 

H 

Non-FSP 

AFROTC  Graduate 

— 

9 

A FA  Graduate 

— 

33 

USAF  Helicopter  Pilot 

1 

— 

Othfer  military  academy 

— 

1 

CT$  Graduate 

404 

351 

USAF  Rated  Officer 

2 

1 

USAF  Son-Rated  Officer 

6 

4 

Hen-USAf  Officer 

21 

46 

Total 

438 

445 

kote.  FSP  «  Flight  Screening  Program;  AFROTC 
•  Air  Force  Reserve  Officer  Training  Cor?s;  AFA  • 
Air  Force  Academy;  OTS  »  Officer  Training  School. 
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Instrumentation 


Embedded  Figures 

This  test  has  been  used  to  examine  the  psychological  factor  of  field  dependence/Independence 
(Goodenough,  ,976;  Witkln,  1949).  It  should  be  noted  that  level  of  field  dependence  has  been 
treated  as  a  personality  characteristic  by  some  researchers  and  as  a  perceptual  ability  by  others. 

In  this  test,  the  subject  was  presented  with  a  simple  geometric  figure  and  two  complex 
figures.  The  task  was  to  decide  which  of  the  two  complex  figures  had  the  simple  figure  embedded 
within  It  and  to  Indicate  a  choice  by  pressing  the  keypad  button  corresponding  to  the  figure. 
Speed  and  accuracy  of  response  were  recorded  on  each  of  the  30  trials. 


Dot  Estimation 


This  test  was  designed  to  study  decisiveness  (Mullins,  1962).  In  both  the  original  and  the 
current  form  of  the  test,  the  subject  was  presented  with  two  boxes,  each  containing  a  random 
display  of  dots.  The  subject  was  Instructed  to  determine,  as  qulc.ly  as  possible,  which  of  the 
two  boxes  contained  more  dots.  One  box  always  contained  one  more  dot.  No  Instructions  were 
given  with  regard  to  counting.  The  rationale  for  the  development  of  the  test  was  that  decisive 

Individuals  would  take  less  time  with  each  Item  and  thus  complete  more  Items  In  a  given  time 

period,  whereas  less  decisive  Individuals  would  sper.J  more  time  counting  and  thus  complete  fewer 
Items  (Mullins,  1962). 

In  this  version  of  the  test,  there  was  a  fixed  time  limit  of  5  minutes  and  55  Items  total. 
Each  box  was  a  3  l/8-1nch  by  3  l/8-1nch  square.  In  the  first  Item,  the  boxes  contained  10  and  11 
dots,  respectively.  With  each  succeeding  Item  (until  the  last  15),  the  number  of  dots  Increased 
by  2  per  screen,  so  that  for  the  last  15  Items,  the  boxes  contained  50  and  51  dots, 
respectively.  The  measures  recorded  were  the  number  of  trials  completed,  the  number  of  correct 

responses,  the  percent  of  correct  responses,  and  the  total  time  spent  on  the  test.  From  these 

measures  was  computed  the  average  time  spent  on  correct  responses. 


Self -Crediting  Word  Knowledge 

This  test  was  designed  to  be  an  objective  measure  of  self-confidence  (Mullins,  1962).  0,i 

each  of  30  trials,  the  subject  was  presented  with  a  ’target"  word  and  five  other  words.  The 
subject's  instructions  were  to  choose  the  synonym  closest  In  meaning  to  that  of  the  target  word. 

The  test  was  divided  Into  three  sections  of  10  questions  each.  With  each  succeeding  block, 
the  Items  became  more  difficult.  The  subject  was  Informed  of  this  fact  and  asked  to  sake  a  bet 
prior  to  each  of  the  three  blocks.  The  average  number  of  points  bet  was  recorded,  as  well  as  the 
percent  correct,  the  average  reaction  time  for  the  correct  trials,  and  a  produti  vector  of 
reaction  time  and  percent  correct. 


Activities  Interest  Inventory 


This  test  was  developed  to  measure  survival  attitudes  ar.d  risk-taking  tendency.  The  subject 
was  presented  with  81  pairs  of  activities  and  was  asked  to  Indicate  i  preference  for  each  pair. 
The  subject  was  told  to  assume  that  he/she  had  the  necessary  ability  to  perform  each  activity. 
The  activity  pairs  forced  the  subject  to  choosr  between  tasks  that  differed  as  to  threat  to 


physical  survival  (sometimes  subtly,  sometimes  not).  The  measures  of  Interest  were  the  number  of 
high-risk  options  chosen  and  the  average  amount  of  time  required  to  choose  between  pairs  of 
activities. 


Risk-Taking 

This  test  was  selected— on  the  basis  of  Its  previous  use  In  a  variety  of  settings  (Slovlc, 
1966)— to  measure  risk-taking  tendencies  when  making  decisions.  Ten  boxes  were  presented  In  two 
rows  of  five  boxes  each.  The  subject  was  told  that  nine  of  the  boxes  contained  a  reward,  and  one 
of  the  boxes  was  a  ’disaster'*  box.  The  subject  was  allowed  to  select  the  boxes  one  at  a  time. 
If  the  selected  box  contained  a  payoff,  the  subject  was  allowed  to  accumulate  a  point  for  that 
“ox.  However,  If  the  subject  chose  the  disaster  box,  all  points  for  that  trial  were  lost. 

The  test  consisted  of  30  trials.  For  12  of  the  trials,  there  was  no  actual  disaster  box. 
The  rationale  for  this  design  was  that  on  the  trials  with  a  disaster  box,  the  “risk  trials,’  th. 
number  of  boxes  selected  was  Impacted  by  chance  selection  of  the  disaster  box.  On  ths  12 
“no-risk*  trials,  however,  the  subject  could  choose  up  to  nine  boxes  without  actt,i-ly 
encountering  the  disaster  box  (as  the  disaster  box  always  appeared  as  the  tenth  box).  '  >  .s,  the 
no-risk  trials  were  designed  to  allow  subjects  to  manifest  their  risk-taking  tendencies 
unaffected  by  chance  selection  of  an  actual  disaster  box. 

The  measures  recorded  on  this  test  were  response  time  and  nursber  of  boxes  chosen  separately 
for  the  risk  and  no-risk  trials. 

UPT  Performance  Criteria 


UPT  final  training  outcome  was  scored  as  a  dichotomous  variable  with  pass  ■  1  and  fall  ■  0. 
Students  who  passed  UPT  received  a  recommendation  from  an  Advanced  Training  Recommendation  Board 
(ATRB)  for  advanced  training  leading  to  an  assignment  either  as  a  FAR  pilot  or  a  TBB  pilot  (FAR  » 
1  and  TTB  •  0).  Final  training  outcome  and  ATRB  recommendation  were  determined.  In  part,  by  a 
subject's  performance  on  six  check  flights  during  UPT. 


Apparatus 

The  BAT  apparatus  (Carretta,  1937)  consisted  of  a  super-microcomputer  built  into  a 
self-contained  unit  with  a  glare  shield  and  side  panels  designed  to  ensure  consistency  of  testing 
sessions.  The  subject  responded  to  the  various  tests  using,  In  combination  or  individually,  a 
two-axis  Joystick  on  the  -Ight  side  of  the  apparatus,  a  single-axis  joystick  on  the  left  side, 
and  a  keyboard  in  the  center  of  the  test  unit.  The  keyboard  included  the  numbers  0  to  9,  an 
“Enable*  key  in  the  center,  and  a  bottom  row  with  ’yes*  and  *no*  keys  and  two  others  labeled 
*$/l*  (for  same/left  responses)  and  *D/R*  (for  dlfferent/right  responses). 


Procedure 


■Host  of  the  subjects  in  the  present  sample  entered  UPT  after  completion  of  Officer  Training 
School  ( 0 T $ ) .  The  other  subjects  case  from  »  variety  of  other  sources  'see  "ibte  1).  Air  Force 
Reserve  Officer  Training  Corps  (AFROTC),  GTS,  and  all  active  duty  USAF  candidates  were 
pre-selected  on  the  Air  Force  Officer  Ckjelifylrtg  Test  (AFOOT).  The  AFOOT  for*  used  with  this 
sample  (Form  0;  Rogers,  Roach,  t  tfegner,  1966)  consisted  of  16  subtests  which  were  combined  to 
for*  five  composite  measures.  All  five  composite  scores  were  available  to  OTS  selection 
boards.  Bv  A'r  Force  Regulation  51-4,  those  candidates  who  were  required  to  take  the  AFOOT  in 
order  to  enter  UPT  had  to  achieve  a  minimum  score  at  the  twenty-^  fth  percertlle  on  the  Pilot 
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composite  and  a  minimum  score  at  the  tenth  percentile  on  the  Navigator-Technical  composite,  and 
the  sum  of  the  two  composite  scores  had  to  be  equal  to  or  greater  than  50. 

Prior  to  entering  UPT,  about  half  the  sample  (_n  -  438)  participated  in  a  14-hour  light 
aircraft  Flight  Screening  Program  (FSP).  This  program  for  pilot  candidates  having  no  previous 
flying  experience  was  designed  to  acquaint  them  with  the  aviation  environment  through  flight 
instruction  in  a  T-41  aircraft.  (Candidates  such  as  OTS  students  with  private  pilot  licenses  and 
Air  Force  Academy  students  who  completed  the  standard  Academy  flight  orientation  program  were 
exempt  from  attending  FSP.)  For  the  students  attending  FSP,  satisfactory  completion  of  the 
course  was  required  to  continue  to  OPT. 

Subjects  were  administered  the  BAT  prior  to  entry  into  UPT.  The  test  battery  consisted  of  14 
tests  that  required  about  3  1/2  hours  to  complete.  After  a  test  adninistrator  initiated  the 
system,  the  test  session  was  self-paced  by  the  subject.  The  test  session  Included  programmed 
breaks  between  tests  to  counteract  fatigue  factors. 

All  candidates  in  the  sample  took  part  in  a  UPT  program  lasting  49  weeks.  Candidates  could 
be  eliminated  from  training  at  any  point  in  the  program  for  a  variety  of  reasons.  Those 
remaining  in  the  program  at  the  forty-second  week  were  considered  by  an  ATRB  for  either  a  FAR 
track  or  a  TTB  track. 


Analysis 


Reliability  analyses  were  conducted  for  all  experimental  test  measures  (on  a  larger  sample 
from  which  the  present  sample  was  drawn).  For  the  present  sample,  descriptive  statistics  were 
computed,  Including  an  intercorrelation  matrix  of  all  measures  of  interest.  These  measures  were 
examined  both  for  the  entire  sample  and  separately  for  UPT  graduates  and  non-graduates.  For  UPT 
graduates  only,  the  data  were  examined  separately  for  those  candidates  who  were  FAR  recommended 
and  those  who  were  TTB  recommended,  A  series  of  regression  analyses  was  conducted.  For  each 
regression,  predictors  were  measures  from  one  of  the  BAT  tests,  with  the  criterion  either  UPT 
pass/fail  or  ATRB  recommendation. 


III.  RESULTS 

Initial  analyses  (reported  elsewhere;  Carretta  4  Slew,  in  preparation)  were  conducted  to  test 
for  the  Internal  consistency  of  the  measures  from  four  of  the  five  tests.  (As  Dot  Estimation  was 
a  speeded  test,  conventional  reliability  analyses  were  not  appropriate.)  For  each  of  the 
measures  reported,  alpha  coefficients  (Cronbach,  1951)  were  considered  acceptable  (greater  than 
.70)  with  the  exception  of  the  measure  of  the  number  of  boxes  chosen  on  risk  trials  in  the 
Risk-Taking  Test.  However,  this  low  reliability  was  to  be  expected  as  there  was  a  chance  element 
determining  when  a  subject  would  select  a  disaster  box  on  any  particular  trial. 

Given  tnat  the  measures  appeared  to  be  reliable,  the  next  step  was  to  determine  their 
validity  in  predicting  pilot  training  outcomes.  A  series  of  regression  analyses  was  conducted, 
two  for  each  of  the  rive  tests.  One  analysis  for  each  test  used  UPT  pass/fail  as  the  dependent 
measure;  for  the  other,  the  dependent  measure  was  the  ATRB  FAR/TTB  recommendation. 

Descriptive  statistics  for  each  measure  and  their  Intercorrelations  are  reported  in  Tables  2 
and  3.  The  multiple  correlation  for  each  test  with  the  two  training  outcome  measures  is  shown  in 
Table  4.  These  correlations  indicate  that  the  only  test  to  predict  UPT  pass/fail  outcome  was  the 
Self-Crediting  Word  Knowledge  Test.  Although  the  correlation  was  significant  (R  »  .14,  _F[4,833] 
-  4.52,  p  <  .01),  the  magnitude  of  the  rt^utlonship  was  modest.  None  of  the  other  four  tests  was 
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correlated  Individually  with  UPT  pass/fall,  nor  were  any  of  tht  five  tests  associated  with  the 
ATRB  reconmendatlon  measure. 


Table  2.  Descriptive  Statistics  for  BAT  Test  Measures  (N  ■  833) 


Time 

Mean 

SO 

Embedded  Figures 

Mean  RT  correct  responses  (msec) 

12,485.76 

4,750.92 

%  correct 

64.56 

13.86 

RTx%  correct 

2,308.45 

68,305.44 

Dot  Estimation 

#  of  trials  completed 

48.72 

12.62 

#  of  correct  responses 

31.31 

7.06 

Total  time  spent  on  test  (sec) 

147.40 

75.27 

Mean  RT,  correct  responses  (sec) 

5.65 

4.93 

%  correct  responses 

66.23 

10.87 

Self-Crediting  Word  Knowledge 

Mean  RT,  correct  responses  (msec) 

8,214.66 

2,055.69 

%  correct 

67.81 

10.27 

RTx%  correct 

-3,095.03 

30,178.40 

8*t 

39.02 

11.42 

>w.,ivities  Interest  Inventory 

#  of  high-risk  choices 

50.39 

9.74 

Mean  RT  (msec) 

4,535.41 

1,144.86 

Risk-Taking 

#  Boxes  chosen,  risk  trials 

4.54 

.82 

Mean  RT,  risk  trials  (msec) 

2,723.65 

1,693.99 

#  boxes  chosen,  no-risk  trials 

6.90 

1.33 

Mean  RT,  no-risk  trials  (msec) 

2,277.67 

1,577.94 

Note.  RT  x%  correct  product  terms  adjusted  for  mean  RT  and  % 
correct  performance. 


The  polnt-biserial  correlations  with  UPT  pass/fail  for  the  measures  that  contributed  to  the 
multiple  correlation  for  Self-Crediting  Word  Knowledge  are  also  shown  in  Table  4.  These 
correlations  indicate  that  the  strongest  relationship  with  training  outcome  was  for  the  mean 
response  time  on  trials  answered  correctly  (_r  [883]  *  .12.  _o_  <  .001).  The  percent  of  correct 
answers  was  also  significantly  but  negatively  correlated  with  UPT  pass/fail  (_r  [883]  a  -.07, 
£  <  .05),  as  was  the  amount  of  points  bet  (_r  [883]  ■  -.06,  _p  <  .05). 

Another  way  to  approach  understanding  these  relationships  between  responses  on  the 
Self-Crediting  Word  Knowledge  Test  and  UPT  pass/fail  is  to  examine  the  means  in  Table  5.  These 
means  indicate  that  subjects  who  passed  UPT  tended  to  take  longer  to  respond  (M  =  8.39s  vs.  _M  » 
7.85s),  had  fewer  correct  responses  (M_  3  67. 8t  vs.  »  68.8 %),  and  bet  fewer  points  on  their 
responses  (M  a  38.5  vs.  M  j  40.1). 
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Table  4.  Correlational  Relationships  Between  BAT  Tests  and  Training  Outcomes 


Test 

UPT  pass/fall 
(N  -  883) 

FAR/TTB 
(N  -  519) 

Embedded  Figures 

Mean  RT,  correct  responses 

.00 

-.03 

l  correct 

-.06 

.05 

RTxi  correct 

.00 

.04 

Multiple  Correlation 

.06 

.08 

Dot  Estimation 

#  of  trials  completed 

o 

o 

• 

-.05 

#  of  correct  responses 

.01 

.00 

Total  time  spent  on  test 

.02 

.03 

Mean  RT,  correct  responses 

.00 

.05 

%  correct  responses 

.01 

.07 

Multiple  Correlation 

.03 

.14 

Self-Crediting  Word  Knowledge 

Mean  RT,  correct  responses 

.12*** 

-.05 

%  correct 

-.07* 

-.03 

RTxi  correct 

.03 

.06 

Bet 

-.06* 

.02 

Multiple  Correlation 

.14** 

.08 

Ac  vlties  Interest  Inventory 

#  of  high-risk  choices 

-.02 

.04 

Mean  RT 

-.06 

-.05 

Mult-lolc  Correlation 

.06 

.07 

Risk-Taking 

#  boxes  cho~<in,  risk  trials 

-.03 

.02 

Mean  RT,  risk  trials 

-.04 

-.01 

#  boxes  chosen,  no-risk  trials 

-.03 

.01 

Mean  RT,  no-risk  trials 

-03 

-.02 

Multiple  Correlation 

.05 

.06 

Note.  RT  ■  Response  Hite;  #  ■  Number;  %  ■  Percent;  kTxi  *  vector  prod¬ 
uct  term.  Significance  ’evels  for  zero-order  correlations  reported  only  when 
multiple  co-relation  is  s1gn1f1c»nt. 


*£<.05.  **£<.01.  ***£<.001. 


Table  5.  Near  Scores  for  Self-Crediting  Word  Knowledge  by 
Training  Outcome 


Test 

measure 

UPT 

fall 

(N  -  284) 

UPT 

pass 

(N  -  599) 

RT  (msec! 

M 

7,854.38 

8,385.48 

SD 

1,753.79 

2,164.79 

%  correct 

M 

6«.S4 

67.81 

SD 

10.03 

10.25 

8et 

M 

40.08 

38.51 

SD 

10. 2r 

11.91 

Note.  Response  time  (RT)  In  milliseconds. 
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For  a  subset  of  the  sample  who  had  data  on  the  Word  Knowledge  subtest  of  the  AFOQT, 
additional  analyses  were  conducted  to  examine  differences  on  the  Self-Crediting  Word  Knowledge 
Items,  controlling  for  overall  verbal  ability.  The  pattern  of  adjusted  means  was  similar  to  that 
for  the  unadjusted  means  (see  Table  6),  although  the  differences  were  smaller.  Note,  however, 
that  graduates  and  ellminees  differed  significantly  In  average  reaction  time,  even  after 
controlling  for  verbal  ability.  These  data  tend  to  suggest  that  candidates  who  successfully 
completed  UPT  tended  to  take  longer  In  their  responses,  which  could  be  Interpreted  as  a 
manifestation  of  a  more  cautious  decision-making  style. 


Table  6.  Mean  Score  for  Self-Crediting  Word  Knowledge  by 
Training  Outcome  Adjusted  for  AFOQT  Word  Knowledge  Scores 


UPT  fall 

UPT  pass 

Measure 

(N  -  259) 

(N  -  488) 

RT  (msec) 

7,919.22 

8,429.67 

%  correct 

68.11 

68.01 

Bet 

39.42 

39.07 

Additional  evidence  for  this  Interpretation  of  the  data  comes  from  comparing  UPT  graduates 
with  non-grads  in  terms  of  their  responses  on  measures  from  the  other  tests  (although  these 
differences  were  not  statistically  significant).  Candidates  who  passed  UPT  selected,  on  the 
average,  fewer  high-risk  choices  on  the  Activities  Interest  Inventory  01  *  50.25  vs.  M_  »  50.70 
[out  of  81  Items]).  UPT  graduates  also  were  "less  Impulsive"  on  the  Dot  Estimation  Test;  that 
Is,  relative  to  UPT  failures,  the  successful  students  completed  fewer  of  the  self-paced  trials 
(J1  *  48.71  vs.  M_  -  48.89),  spent  more  time  on  the  test  (M  ■  148.26s  vs.  M^  ■  146.11s),  and  spent 
more  time  on  correct  responses  (M  ■  56.62  vs.  _M  «  56.42).  The  UPT  graduates  also  answered  more 
of  the  Items  correctly  01  «  66.33X  vs.  M.  ■  66.25%) ;  although  the  differences  on  the  Dot 
Estimation  measures  between  UPT  graduates  and  ellminees  were  not  statistically  significant,  they 
are  consistent  with  the  Interpretation  of  responses  to  the  Self-Crediting  Word  Knowledge  Test 
that  UPT  graduates  were  more  cautious  and  deliberate,  less  Impulsive  and  hasty,  In  their 
approaches  to  the  experimental  tests  than  were  the  UPT  failures. 

IV.  DISCUSSION 

Each  of  the  five  BAT  tests  exhibited  acceptable  reliability  for  use  as  selection  devices. 
However,  none  of  them  was  related  statistically  to  both  measures  of  training  performance 
(graduation/el iml nation  and  advanced  training  assignment).  There  are  several  explanations  for 
the  poor  predictive  utility  demonstrated  by  these  tests.  One  explanation  Is  that  the  tests  may 
not  have  been  measuring  the  characteristics  that  they  were  designed  to  measure  (l.e.,  poor 
construct  validity).  Although  all  but  one  of  the  tests  was  adapted  from  a  previously  validated 
paper-and-pencll  test,  no  subjects  were  given  both  the  original  form  and  the  computerized  version 
of  the  same  test.  That  Is,  some  facet  of  computer  administration  of  the  test  may  have  changed 
the  characteristics  of  the  test. 

Another  possible  explanation  for  the  low  validity  coefficients  for  these  tests  Is  a 
restriction  In  the  range  of  subject  variability.  In  terms  of  test  performance.  That  Is,  the 
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candidates  who  entered  UPT  may  have  been  a  fairly  homogeneous  group,  as  most  already  had  been 
screened  on  the  AFOQT  and  some  form  of  flight  training.  In  some  fashion,  those  "gates*  may  have 
selected  out  candidates  who  differed  from  those  actually  entering  UPT,  In  terms  of  risk-taking 
behaviors  and  attitudes,  self-confidence,  and  the  other  characteristics  that  these  tests  were 
designed  to  measure. 

Finally,  these  tests  may  be  accurately  assessing  what  they  purport  to  measure,  but  those 
characteristics  may  not  be  relevant  to  predicting  success  In  pilot  training.  Even  with  the  one 
test  demonstrating  modest  validity  In  the  present  sample,  Self-Crediting  Word  Knowledge,  the 
average  performance  difference  between  passes  and  non-graduates  was  fairly  small  (see  Table  6). 
However,  In  analyses  reported  elsewhere  {Carretta,  1987),  this  test  continued  to  be  a  significant 
predictor  of  UPT  performance  when  Included  in  a  prediction  model  that  consisted  of  a  number  of 
psychomotor  and  cognitive  skills  tests,  the  types  of  tests  that  have  been  used  more  traditionally 
for  pilot  selection.  Thus,  It  appears  that  the  Self-Crediting  Word  Knowledge  Test  may  contribute 
unique  predictive  validity  to  a  selection  system  for  pilot  candidates. 


V.  CONCLUSION 

Future  efforts  will  be  directed  at  Improving  and  refining  the  present  version  of  the 
Self-Crediting  Word  Knowledge  Test.  Other  research  efforts  currently  underway  are  designed  to 
determine  the  test's  construct  validity.  In  order  to  develop  a  better  Idea  of  exactly  what  the 
test  Is  measuring,  more  traditional  personality  tests  of  characteristics  such  as  self-confidence 
(e.g.,  Spence,  Helmrelch,  S  Holahan,  1979)  are  being  administered  to  Air  Force  pilots  with 
varying  levels  of  training  experience  who  have  been  tested  previously  on  the  BAT.  By  examining 
relationships  between  the  traditional,  validated  tests  of  self-confidence  with  the  Self-Crediting 
Word  Knowledge  measures,  a  better  understanding  of  the  characteristics  of  the  test  and  how  It  can 
be  Implemented  In  future  models  for  Air  Force  pilot  selection  and  classification  can  be  developed. 
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